Alignment of brain embeddings and artificial contextual embeddings in natural language points to common geometric patterns.

Contextual embeddings, derived from deep language models (DLMs), provide a continuous vectorial representation of language. This embedding space differs fundamentally from the symbolic representations posited by traditional psycholinguistics. We hypothesize that language areas in the human brain, similar to DLMs, rely on a continuous embedding space to represent language. To test this hypothesis, we densely record the neural activity patterns in the inferior frontal gyrus (IFG) of three participants using dense intracranial arrays while they listened to a 30-minute podcast. From these fine-grained spatiotemporal neural recordings, we derive a continuous vectorial representation for each word (i.e., a brain embedding) in each patient. Using stringent zero-shot mapping we demonstrate that brain embeddings in the IFG and the DLM contextual embedding space have common geometric patterns. The common geometric patterns allow us to predict the brain embedding in IFG of a given left-out word based solely on its geometrical relationship to other non-overlapping words in the podcast. Furthermore, we show that contextual embeddings capture the geometry of IFG embeddings better than static word embeddings. The continuous brain embedding space exposes a vector-based neural code for natural language processing in the human brain.

Three-dimensional histology reveals dissociable human hippocampal long-axis gradients of Alzheimer's pathology.

INTRODUCTION: Three-dimensional (3D) histology analyses are essential to overcome sampling variability and understand pathological differences beyond the dissection axis. We present Path2MR, the first pipeline allowing 3D reconstruction of sparse human histology without a magnetic resonance imaging (MRI) reference. We implemented Path2MR with post-mortem hippocampal sections to explore pathology gradients in Alzheimer's disease. METHODS: Blockface photographs of brain hemisphere slices are used for 3D reconstruction, from which an MRI-like image is generated using machine learning. Histology sections are aligned to the reconstructed hemisphere and subsequently to an atlas in standard space. RESULTS: Path2MR successfully registered histological sections to their anatomic position along the hippocampal longitudinal axis. Combined with histopathology quantification, we found an expected peak of tau pathology at the anterior end of the hippocampus, whereas amyloid-beta (Aß) displayed a quadratic anterior-posterior distribution. CONCLUSION: Path2MR, which enables 3D histology using any brain bank data set, revealed significant differences along the hippocampus between tau and Aß. HIGHLIGHTS: Path2MR enables three-dimensional (3D) brain reconstruction from blockface dissection photographs. This pipeline does not require dense specimen sampling or a subject-specific magnetic resonance (MR) image. Anatomically consistent mapping of hippocampal sections was obtained with Path2MR. Our analyses revealed an anterior-posterior gradient of hippocampal tau pathology. In contrast, the peak of amyloid-beta (Aß) deposition was closer to the hippocampal body.

Decentralized Mixed Effects Modeling in COINSTAC.

Performing group analysis on magnetic resonance imaging (MRI) data with linear mixed-effects (LME) models is challenging due to its large dimensionality and inherent multi-level covariance structure. In addition, as large-scale collaborative projects become commonplace in neuroimaging, data must increasingly be stored and analyzed from different locations. In such settings, substantial overhead can occur in terms of data transfer and coordination between participating research groups. In some cases, data cannot be pooled together due to privacy or regulatory concerns. In this work, we propose a decentralized LME model to perform a large-scale analysis of data from different collaborations without data pooling. This method is efficient as it overcomes the hurdles of data sharing and has lower bandwidth and memory requirements for analysis than the centralized modeling approach. We evaluate our model using features extracted from structural magnetic resonance imaging (sMRI) data. Results highlight gray matter reductions in the temporal lobe/insula and medial frontal regions in schizophrenia, consistent with prior studies. Our analysis also demonstrates that decentralized LME models achieve similar performance compared to the models trained with all the data in one location. We also implement the decentralized LME approach in COINSTAC, an open source, decentralized platform for federating neuroimaging analysis, providing an easy to use tool for dissemination to the neuroimaging community.

Machine learning of dissection photographs and surface scanning for quantitative 3D neuropathology.

We present open-source tools for 3D analysis of photographs of dissected slices of human brains, which are routinely acquired in brain banks but seldom used for quantitative analysis. Our tools can: (i) 3D reconstruct a volume from the photographs and, optionally, a surface scan; and (ii) produce a high-resolution 3D segmentation into 11 brain regions per hemisphere (22 in total), independently of the slice thickness. Our tools can be used as a substitute for ex vivo magnetic resonance imaging (MRI), which requires access to an MRI scanner, ex vivo scanning expertise, and considerable financial resources. We tested our tools on synthetic and real data from two NIH Alzheimer's Disease Research Centers. The results show that our methodology yields accurate 3D reconstructions, segmentations, and volumetric measurements that are highly correlated to those from MRI. Our method also detects expected differences between post mortem confirmed Alzheimer's disease cases and controls. The tools are available in our widespread neuroimaging suite "FreeSurfer" ( https://surfer.nmr.mgh.harvard.edu/fswiki/PhotoTools ).

Three-dimensional histology reveals dissociable human hippocampal long axis gradients of Alzheimer's pathology.

INTRODUCTION: Three-dimensional (3D) histology analyses are essential to overcome sampling variability and understand pathological differences beyond the dissection axis. We present Path2MR, the first pipeline allowing 3D reconstruction of sparse human histology without an MRI reference. We implemented Path2MR with post-mortem hippocampal sections to explore pathology gradients in Alzheimer's Disease. METHODS: Blockface photographs of brain hemisphere slices are used for 3D reconstruction, from which an MRI-like image is generated using machine learning. Histology sections are aligned to the reconstructed hemisphere and subsequently to an atlas in standard space. RESULTS: Path2MR successfully registered histological sections to their anatomical position along the hippocampal longitudinal axis. Combined with histopathology quantification, we found an expected peak of tau pathology at the anterior end of the hippocampus, while amyloid-ß displayed a quadratic anterior-posterior distribution. CONCLUSION: Path2MR, which enables 3D histology using any brain bank dataset, revealed significant differences along the hippocampus between tau and amyloid-ß.

Enhancing collaborative neuroimaging research: introducing COINSTAC Vaults for federated analysis and reproducibility.

Collaborative neuroimaging research is often hindered by technological, policy, administrative, and methodological barriers, despite the abundance of available data. COINSTAC (The Collaborative Informatics and Neuroimaging Suite Toolkit for Anonymous Computation) is a platform that successfully tackles these challenges through federated analysis, allowing researchers to analyze datasets without publicly sharing their data. This paper presents a significant enhancement to the COINSTAC platform: COINSTAC Vaults (CVs). CVs are designed to further reduce barriers by hosting standardized, persistent, and highly-available datasets, while seamlessly integrating with COINSTAC's federated analysis capabilities. CVs offer a user-friendly interface for self-service analysis, streamlining collaboration, and eliminating the need for manual coordination with data owners. Importantly, CVs can also be used in conjunction with open data as well, by simply creating a CV hosting the open data one would like to include in the analysis, thus filling an important gap in the data sharing ecosystem. We demonstrate the impact of CVs through several functional and structural neuroimaging studies utilizing federated analysis showcasing their potential to improve the reproducibility of research and increase sample sizes in neuroimaging studies.

Enhancing Collaborative Neuroimaging Research: Introducing COINSTAC Vaults for Federated Analysis and Reproducibility.

Collaborative neuroimaging research is often hindered by technological, policy, administrative, and methodological barriers, despite the abundance of available data. COINSTAC is a platform that successfully tackles these challenges through federated analysis, allowing researchers to analyze datasets without publicly sharing their data. This paper presents a significant enhancement to the COINSTAC platform: COINSTAC Vaults (CVs). CVs are designed to further reduce barriers by hosting standardized, persistent, and highly-available datasets, while seamlessly integrating with COINSTAC's federated analysis capabilities. CVs offer a user-friendly interface for self-service analysis, streamlining collaboration and eliminating the need for manual coordination with data owners. Importantly, CVs can also be used in conjunction with open data as well, by simply creating a CV hosting the open data one would like to include in the analysis, thus filling an important gap in the data sharing ecosystem. We demonstrate the impact of CVs through several functional and structural neuroimaging studies utilizing federated analysis showcasing their potential to improve the reproducibility of research and increase sample sizes in neuroimaging studies.

Fine temporal brain network structure modularizes and localizes differently in men and women: insights from a novel explainability framework.

Deep learning has become an effective tool for classifying biological sex based on functional magnetic resonance imaging (fMRI). However, research on what features within the brain are most relevant to this classification is still lacking. Model interpretability has become a powerful way to understand "black box" deep-learning models, and select features within the input data that are most relevant to the correct classification. However, very little work has been done employing these methods to understand the relationship between the temporal dimension of functional imaging signals and the classification of biological sex. Consequently, less attention has been paid to rectifying problems and limitations associated with feature explanation models, e.g. underspecification and instability. In this work, we first provide a methodology to limit the impact of underspecification on the stability of the measured feature importance. Then, using intrinsic connectivity networks from fMRI data, we provide a deep exploration of sex differences among functional brain networks. We report numerous conclusions, including activity differences in the visual and cognitive domains and major connectivity differences.

Federated Analysis in COINSTAC Reveals Functional Network Connectivity and Spectral Links to Smoking and Alcohol Consumption in Nearly 2,000 Adolescent Brains.

With the growth of decentralized/federated analysis approaches in neuroimaging, the opportunities to study brain disorders using data from multiple sites has grown multi-fold. One such initiative is the Neuromark, a fully automated spatially constrained independent component analysis (ICA) that is used to link brain network abnormalities among different datasets, studies, and disorders while leveraging subject-specific networks. In this study, we implement the neuromark pipeline in COINSTAC, an open-source neuroimaging framework for collaborative/decentralized analysis. Decentralized exploratory analysis of nearly 2000 resting-state functional magnetic resonance imaging datasets collected at different sites across two cohorts and co-located in different countries was performed to study the resting brain functional network connectivity changes in adolescents who smoke and consume alcohol. Results showed hypoconnectivity across the majority of networks including sensory, default mode, and subcortical domains, more for alcohol than smoking, and decreased low frequency power. These findings suggest that global reduced synchronization is associated with both tobacco and alcohol use. This proof-of-concept work demonstrates the utility and incentives associated with large-scale decentralized collaborations spanning multiple sites.

Interpreting models interpreting brain dynamics.

Brain dynamics are highly complex and yet hold the key to understanding brain function and dysfunction. The dynamics captured by resting-state functional magnetic resonance imaging data are noisy, high-dimensional, and not readily interpretable. The typical approach of reducing this data to low-dimensional features and focusing on the most predictive features comes with strong assumptions and can miss essential aspects of the underlying dynamics. In contrast, introspection of discriminatively trained deep learning models may uncover disorder-relevant elements of the signal at the level of individual time points and spatial locations. Yet, the difficulty of reliable training on high-dimensional low sample size datasets and the unclear relevance of the resulting predictive markers prevent the widespread use of deep learning in functional neuroimaging. In this work, we introduce a deep learning framework to learn from high-dimensional dynamical data while maintaining stable, ecologically valid interpretations. Results successfully demonstrate that the proposed framework enables learning the dynamics of resting-state fMRI directly from small data and capturing compact, stable interpretations of features predictive of function and dysfunction.

Decentralized Brain Age Estimation Using MRI Data.

Recent studies have demonstrated that neuroimaging data can be used to estimate biological brain age, as it captures information about the neuroanatomical and functional changes the brain undergoes during development and the aging process. However, researchers often have limited access to neuroimaging data because of its challenging and expensive acquisition process, thereby limiting the effectiveness of the predictive model. Decentralized models provide a way to build more accurate and generalizable prediction models, bypassing the traditional data-sharing methodology. In this work, we propose a decentralized method for biological brain age estimation using support vector regression models and evaluate it on three different feature sets, including both volumetric and voxelwise structural MRI data as well as resting functional MRI data. The results demonstrate that our decentralized brain age regression models can achieve similar performance compared to the models trained with all the data in one location.

Shared computational principles for language processing in humans and deep language models.

Departing from traditional linguistic models, advances in deep learning have resulted in a new type of predictive (autoregressive) deep language models (DLMs). Using a self-supervised next-word prediction task, these models generate appropriate linguistic responses in a given context. In the current study, nine participants listened to a 30-min podcast while their brain responses were recorded using electrocorticography (ECoG). We provide empirical evidence that the human brain and autoregressive DLMs share three fundamental computational principles as they process the same natural narrative: (1) both are engaged in continuous next-word prediction before word onset; (2) both match their pre-onset predictions to the incoming word to calculate post-onset surprise; (3) both rely on contextual embeddings to represent words in natural contexts. Together, our findings suggest that autoregressive DLMs provide a new and biologically feasible computational framework for studying the neural basis of language.

Federated Analysis of Neuroimaging Data: A Review of the Field.

The field of neuroimaging has embraced sharing data to collaboratively advance our understanding of the brain. However, data sharing, especially across sites with large amounts of protected health information (PHI), can be cumbersome and time intensive. Recently, there has been a greater push towards collaborative frameworks that enable large-scale federated analysis of neuroimaging data without the data having to leave its original location. However, there still remains a need for a standardized federated approach that not only allows for data sharing adhering to the FAIR (Findability, Accessibility, Interoperability, Reusability) data principles, but also streamlines analyses and communication while maintaining subject privacy. In this paper, we review a non-exhaustive list of neuroimaging analytic tools and frameworks currently in use. We then provide an update on our federated neuroimaging analysis software system, the Collaborative Informatics and Neuroimaging Suite Toolkit for Anonymous Computation (COINSTAC). In the end, we share insights on future research directions for federated analysis of neuroimaging data.

Federation of Brain Age Estimation in Structural Neuroimaging Data.

Brain age estimation is a widely used approach to evaluate the impact of various neurological or psychiatric brain disorders on the brain developmental or aging process. Current studies show that neuroimaging data can be used to predict brain age, as it captures structural and functional changes that the brain undergoes during development and the aging process. A robust brain age prediction model not only has the potential in assisting early diagnosis of brain disorders but also helps in monitoring and evaluating effects of a treatment. Although access to large amounts of data helps build better models and validate their effectiveness, researchers often have limited access to brain data because of its challenging and expensive acquisition process. This data is not always sharable due to privacy restrictions. Decentralized models provide a way which does not require data exchange between the multiple involved groups. In this work, we propose a decentralized approach for brain age prediction and evaluate our models using features extracted from structural MRI data. Results demonstrate that our decentralized brain age model achieves similar performance compared to the models trained with all the data in one location.

Dynamic connectivity predicts acute motor impairment and recovery post-stroke.

Thorough assessment of cerebral dysfunction after acute lesions is paramount to optimize predicting clinical outcomes. We here built random forest classifier-based prediction models of acute motor impairment and recovery post-stroke. Predictions relied on structural and resting-state fMRI data from 54 stroke patients scanned within the first days of symptom onset. Functional connectivity was estimated via static and dynamic approaches. Motor performance was phenotyped in the acute phase and 6 months later. A model based on the time spent in specific dynamic connectivity configurations achieved the best discrimination between patients with and without motor impairments (out-of-sample area under the curve, 95% confidence interval: 0.67 ± 0.01). In contrast, patients with moderate-to-severe impairments could be differentiated from patients with mild deficits using a model based on the variability of dynamic connectivity (0.83 ± 0.01). Here, the variability of the connectivity between ipsilesional sensorimotor cortex and putamen discriminated the most between patients. Finally, motor recovery was best predicted by the time spent in specific connectivity configurations (0.89 ± 0.01) in combination with the initial impairment. Here, better recovery was linked to a shorter time spent in a functionally integrated configuration. Dynamic connectivity-derived parameters constitute potent predictors of acute impairment and recovery, which, in the future, might inform personalized therapy regimens to promote stroke recovery.

Modular and state-relevant functional network connectivity in high-frequency eyes open vs eyes closed resting fMRI data.

BACKGROUND: Resting-state fMRI (rs-fMRI) is employed to assess "functional connections" of signal between brain regions. However, multiple rs-fMRI paradigms and data-filtering strategies have been used, highlighting the need to explore BOLD signal across the spectrum. Rs-fMRI data is typically filtered at frequencies ranging between 0.008∼0.2 Hz to mitigate nuisance signal (e.g. cardiac, respiratory) and maximize neuronal BOLD signal. However, some argue neuronal BOLD signal may be parsed at higher frequencies. NEW METHOD: To assess the contributions of rs-fMRI along the BOLD spectra on functional network connectivity (FNC) matrices, a spatially constrained independent component analysis (ICA) was performed at seven different frequency "bins", after which FNC values and FNC measures of matrix-randomness were assessed using linear mixed models. RESULTS: Results show FNCs at higher-frequency bins display similar randomness to those from the typical frequency bins (0.01-0.15), while the largest values are in the 0.31-0.46 Hz bin. Eyes open (EO) vs eyes closed (EC) comparison found EC was less random than EO across most frequency bins. Further, FNC was greater in EC across auditory and cognitive control pairings while EO values were greater in somatomotor, visual, and default mode FNC. COMPARISON WITH EXISTING METHODS: Effect sizes of frequency and resting-state paradigm vary from small to large, but the most notable results are specific to frequency ranges and resting-state paradigm with artifacts like motion displaying negligible effect sizes. CONCLUSIONS: These suggest unique information may be derived from FNC matrices across frequencies and paradigms, but additional data is necessary prior to any definitive conclusions.

Decentralized Multisite VBM Analysis During Adolescence Shows Structural Changes Linked to Age, Body Mass Index, and Smoking: a COINSTAC Analysis.

There has been an upward trend in developing frameworks that enable neuroimaging researchers to address challenging questions by leveraging data across multiple sites all over the world. One such open-source framework is the Collaborative Informatics and Neuroimaging Suite Toolkit for Anonymous Computation (COINSTAC) that works on Windows, macOS, and Linux operating systems and leverages containerized analysis pipelines to analyze neuroimaging data stored locally across multiple physical locations without the need for pooling the data at any point during the analysis. In this paper, the COINSTAC team partnered with a data collection consortium to implement the first-ever decentralized voxelwise analysis of brain imaging data performed outside the COINSTAC development group. Decentralized voxel-based morphometry analysis of over 2000 structural magnetic resonance imaging data sets collected at 14 different sites across two cohorts and co-located in different countries was performed to study the structural changes in brain gray matter which linked to age, body mass index (BMI), and smoking. Results produced by the decentralized analysis were consistent with and extended previous findings in the literature. In particular, a widespread cortical gray matter reduction (resembling a 'default mode network' pattern) and hippocampal increase with age, bilateral increases in the hypothalamus and basal ganglia with BMI, and cingulate and thalamic decreases with smoking. This work provides a critical real-world test of the COINSTAC framework in a "Large-N" study. It showcases the potential benefits of performing multivoxel and multivariate analyses of large-scale neuroimaging data located at multiple sites.

Acute ischaemic stroke alters the brain's preference for distinct dynamic connectivity states.

Acute ischaemic stroke disturbs healthy brain organization, prompting subsequent plasticity and reorganization to compensate for the loss of specialized neural tissue and function. Static resting state functional MRI studies have already furthered our understanding of cerebral reorganization by estimating stroke-induced changes in network connectivity aggregated over the duration of several minutes. In this study, we used dynamic resting state functional MRI analyses to increase temporal resolution to seconds and explore transient configurations of motor network connectivity in acute stroke. To this end, we collected resting state functional MRI data of 31 patients with acute ischaemic stroke and 17 age-matched healthy control subjects. Stroke patients presented with moderate to severe hand motor deficits. By estimating dynamic functional connectivity within a sliding window framework, we identified three distinct connectivity configurations of motor-related networks. Motor networks were organized into three regional domains, i.e. a cortical, subcortical and cerebellar domain. The dynamic connectivity patterns of stroke patients diverged from those of healthy controls depending on the severity of the initial motor impairment. Moderately affected patients (n = 18) spent significantly more time in a weakly connected configuration that was characterized by low levels of connectivity, both locally as well as between distant regions. In contrast, severely affected patients (n = 13) showed a significant preference for transitions into a spatially segregated connectivity configuration. This configuration featured particularly high levels of local connectivity within the three regional domains as well as anti-correlated connectivity between distant networks across domains. A third connectivity configuration represented an intermediate connectivity pattern compared to the preceding two, and predominantly encompassed decreased interhemispheric connectivity between cortical motor networks independent of individual deficit severity. Alterations within this third configuration thus closely resembled previously reported ones originating from static resting state functional MRI studies post-stroke. In summary, acute ischaemic stroke not only prompted changes in connectivity between distinct networks, but it also caused characteristic changes in temporal properties of large-scale network interactions depending on the severity of the individual deficit. These findings offer new vistas on the dynamic neural mechanisms underlying acute neurological symptoms, cortical reorganization and treatment effects in stroke patients.

Decentralized distribution-sampled classification models with application to brain imaging.

BACKGROUND: In this age of big data, certain models require very large data stores in order to be informative and accurate. In many cases however, the data are stored in separate locations requiring data transfer between local sites which can cause various practical hurdles, such as privacy concerns or heavy network load. This is especially true for medical imaging data, which can be constrained due to the health insurance portability and accountability act (HIPAA) which provides security protocols for medical data. Medical imaging datasets can also contain many thousands or millions of features, requiring heavy network load. NEW METHOD: Our research expands upon current decentralized classification research by implementing a new singleshot method for both neural networks and support vector machines. Our approach is to estimate the statistical distribution of the data at each local site and pass this information to the other local sites where each site resamples from the individual distributions and trains a model on both locally available data and the resampled data. The model for each local site produces its own accuracy value which are then averaged together to produce the global average accuracy. RESULTS: We show applications of our approach to handwritten digit classification as well as to multi-subject classification of brain imaging data collected from patients with schizophrenia and healthy controls. Overall, the results showed comparable classification accuracy to the centralized model with lower network load than multishot methods. COMPARISON WITH EXISTING METHODS: Many decentralized classifiers are multishot, requiring heavy network traffic. Our model attempts to alleviate this load while preserving prediction accuracy. CONCLUSIONS: We show that our proposed approach performs comparably to a centralized approach while minimizing network traffic compared to multishot methods.

Decentralized Analysis of Brain Imaging Data: Voxel-Based Morphometry and Dynamic Functional Network Connectivity.

In the field of neuroimaging, there is a growing interest in developing collaborative frameworks that enable researchers to address challenging questions about the human brain by leveraging data across multiple sites all over the world. Additionally, efforts are also being directed at developing algorithms that enable collaborative analysis and feature learning from multiple sites without requiring the often large data to be centrally located. In this paper, we propose two new decentralized algorithms: (1) A decentralized regression algorithm for performing a voxel-based morphometry analysis on structural magnetic resonance imaging (MRI) data and, (2) A decentralized dynamic functional network connectivity algorithm which includes decentralized group ICA and sliding-window analysis of functional MRI data. We compare results against those obtained from their pooled (or centralized) counterparts on the same data i.e., as if they are at one site. Results produced by the decentralized algorithms are similar to the pooled-case and showcase the potential of performing multi-voxel and multivariate analyses of data located at multiple sites. Such approaches enable many more collaborative and comparative analysis in the context of large-scale neuroimaging studies.